Medical Imaging Device With Multiple Imaging Modes

ABSTRACT

Improved fluorescent imaging and other sensor data imaging processes, including hyperspectral imaging, devices, and systems are provided to enhance endoscopes with multiple wavelength capabilities and providing sequential imaging and display. A first optical device is provided for endoscopy imaging in a white light and a fluoresced light mode with an imaging unit including one or more image sensors. A mechanism in the first optical device to automatically adjust the focus of the first optical device using one or more deformable, variable-focus lenses, wherein the automatic focus adjustment compensates for a chromatic focal difference between the light collected at distinct wavelength bands caused by the dispersive or diffractive properties of the optical materials or optical design employed in the construction of the first or second optical devices, or both. Further variable spectrum imaging is enhanced with the use of adjustable spectral filters.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority to German Application No.102020105459.9 filed Mar. 2, 2020, entitled, “MEDICAL IMAGING DEVICEWITH MULTIPLE IMAGING MODES” and is incorporated herein by reference.

FIELD OF THE INVENTION

The present invention relates to an observation instrument, particularlyone in the field of endoscopy, and to a distally placed video imagerarrangement for such an observation instrument. The invention relatesgenerally to the field of image capture and more specifically to medicalimaging camera heads, endoscopes, video endoscopes and camera designscapable of imaging in multiple spectra.

BACKGROUND OF THE INVENTION

Endoscopes for medical or non-medical applications generally employ anelongate shaft configured for being introduced into an internal cavityof a human or animal body or another object to be examined. In a distal(i.e. distant from a user) end section of the shaft imaging optics, suchas an objective lens, may be arranged for collecting image light andgenerating an image of a subject scene in the cavity of the body orobject. Further, the endoscope may have a handle attached to a proximal(i.e. close to a user) end section of the shaft. In video endoscopes,which also are denoted electronic endoscopes, the captured endoscopicimage is picked up by one or more electronic image sensors. The imagesensor (or sensors) may be located in the distal end of the shaft, as iscommon in endoscopes frequently referred to as “Chip on the tip” (COTT)endoscopes, or may be located in a camera head element, to which theshaft is attached, the collected image light being generally relayedfrom the distal end to the proximal end by optical elements, such as rodlenses.

Certain endoscopic observations may employ fluorescing agents orauto-fluorescence to better examine tissue. A fluorescing agent such asa dye may be injected or otherwise administered to tissue and anexcitation light directed toward the tissue. Responsive to theexcitation light, the fluorescing agent fluoresces (emits lighttypically at a longer wavelength than the excitation light), allowing animage sensor to detect the emitted light that is often in a wavelengthnot visible to the human eye. The detected images may be examined toindicate the concentration of fluorescing agent in the observed tissue.Further, a phenomenon known as autofluorescence may occur in whichtissue fluoresces under certain conditions without a fluorescing agent.Such light can be detected as well. Images based on detected fluorescedlight, known as “fluorescence imaging” (FI), are useful in medicaldiagnosis, testing, and other scientific fields.

In related technologies, European patent application EP3420880A1discloses a fluorescence imaging scope with variable focusing structuresfor in-focus capture of image streams in the visible and infraredspectra ranges, obviating, thereby, the need to manually refocus theimage subsequent to switching the illumination and viewing spectrum.European application EP1327414B1 describes a device for the imagingdiagnosis of tissue, in which a degree of transmission of anillumination light can be adjusted by means of a variable spectralfilter.

As mentioned above, a typical prior art endoscope generally includes afirst imaging lens (e.g., an objective) which may be followed by aseries of carrier lenses (e.g., relays) which capture and transmit anoptical image from inside an enclosed area, generally of a human oranimal body, to a region outside of the body. The proximal end of theendoscope may be attached, via direct coupling or an adaptor, to acamera head or an eye-piece for viewing. The camera head may includelenses for receiving the optical image and forming an optical image ontothe image sensor. The digital image captured by the image sensor canthen be transmitted to a camera control unit (CCU) or other similarimaging unit or module for analysis and display.

State of the art endoscopes used for fluorescent imaging (FI)applications, and particularly indocyanine green (ICG) applications, arefrequently designed and deployed primarily for visible light imagery. Toperform FI imaging, such scopes may employ an appropriate optical filterto block the stimulus light and transmit fluoresced light. Since theseendoscopes are designed for use with visible light (wavelengths ofapproximately 450-650 nm), the infrared fluorescence (generally 800-900nm) is focused in a different plane than the visible light. Therefore,in addition to adding a filter, the user must refocus when switchingbetween visible light mode and fluorescence mode. Focal differencesexist because the endoscopes are not chromatically corrected for theinfrared where certain fluorescence bands (particularly those associatedwith ICG) are located. Such differences, even in the face of commoncorrection techniques, often result in a signal to noise ratio of FIimaging being low, resulting in poor quality FI images. Given thedispersion characteristics of optical elements used in the endoscope'soptical channel, such as rod lenses, correcting these issues may bedifficult or expensive. For example, when autofocus algorithms areemployed, the algorithm is frequently slow to apply the correction.

It is therefore an object of the present invention to provide anendoscope in which the above mentioned drawbacks are largely avoided. Inparticular, it is an object of the invention to provide devices andmethods that enable an endoscopic system to compensate for the endoscopecharacteristics when detection mode is switched between white lightfluorescence imaging, and to provide sequential imaging, processing anddisplay of images captured in multiple spectra.

These objects are met by a medical imaging system according to claim 1,and by image analysis methods according to claims 16 and 17. Subsequentdependent claims elucidate further advantages of the invention.

The present invention, according to an aspect of the invention, relatesto a medical imaging system with medical imaging device, generally anendoscope. The medical imaging device is operable to capture a firstimage of a subject scene in first finite wavelength band of light, andcapture a second image of the subject scene in a second finitewavelength band of light. The first and second wavelength bands are notthe same, that is, while there may be some overlap in the wavelengths ofeach band, they are not identical in range and/or value. The imagingdevice includes a first imaging lens to capture light from anilluminated subject scene and an imaging unit with one or more imagesensors that may capture images in the first wavelength band and thesecond wavelength band. The medical imaging device also includes adeformable, variable focal length lens located upstream of the one ormore of the image sensors. The deformable lens automatically adjusts, bymeans of adjustment control circuitry, the focus of the medical imagingdevice to compensate for chromatic focal difference between the lightreceived by the one or more image sensors at the first wavelength bandand light received by the one or more image sensors at the secondwavelength band, the chromatic focal difference being a result of thedispersive or diffractive properties of optical materials or opticaldesign employed in an assembly construction of the medical imagingdevice, resulting in images being captured in-focus regardless of thewavelength band of the collected image light. The medical imaging systemalso includes an image processor that receives a first image from theimaging unit and a second image from the imaging unit.

The image processor according to the present invention may be operativeto form a composite image by combining portions of the first image andportions of the second image. Images collected in a wavelength regionoutside of the visible spectrum, may have portions thereof displayed, onan image display, in false color representations in the composite image,such as a composite overlay, with the non-visible light image overlaid,in false color, over the visible light image. According to preferredembodiments of the invention multiple frames at the first wavelengthband and the second wavelength band may be displayed (or stored in asystem memory) sequentially, resulting in a real time, or near-realtime, video stream. In a most preferred embodiment, the frame rate ofthe video stream will be at least 10 frames per second (FPS), and morepreferably 24 or 30 FPS, which are industry standards.

In another aspect of the invention, the light collected by the medicalimaging device, such as a detachable or affixed endoscope, may be splitupstream from the imaging unit, and the imaging unit may contain onesensor for each resultant split beam. In some embodiments, the beamsplitter splits the incoming beam by wavelength, the resulting splitbeams having differing, non-overlapping wavelength bands. Along at leastone of the resulting optical paths will be positioned the deformablelens, but in some embodiments both beam paths may have a deformable,variable focal length lens.

In another embodiment of the invention, which may be used in conjunctionwith other embodiments, the image sensor or sensors may include spectralfilters as integral elements thereof, that is, individual pixels and/orgroups of pixels of the image sensor may be covered by spectral filters.

In another aspect of the invention, the medical imaging device mayinclude one or more variable band spectral filters, the spectral bandpassed by each spectral filter being determined by its angular positionalong its respective optical channel. In preferred embodiments with abeam splitter, each optical channel, may contain both a variable bandspectral filter and a deformable, variable focal length lens, thedeformable lens automatically correcting the focal length of the opticalchannel in response to the selected wavelength band passed by therespective filter, resulting in in-focus images collected by therespective image sensor for each collected image at each transmittedspectral band. In some embodiments, the position of the spectral filtersmay be controlled by a respective actuator.

According to another aspect of the invention, the medical imaging deviceis employed to generate sequential, composite image frames of a subjectscene, where the composite image frame, generated by an image processor,is composed of an image captured in a visible spectrum and an imagecaptured in a non-visible spectrum, the non-visible spectrum image maybe represented on a display as a false-colored image overlaid on thevisible spectrum image. In this aspect of the invention, an image scenemay be illuminated with a first wavelength band of light, and the imagelight from the scene is collected by means such as a first imaging lens.The collected light passes through a deformable, variable-focal lengthlens and onto an image sensor. The focal length of the deformable lensis adjusted such that an image captured by image sensor is in-focus, andthe captured image data for the visible spectrum image is transmitted toan image processor. Subsequently, the subject scene is illuminated witha second wavelength band of light selected to cause at least a portionof the subject scene to fluoresce, the resulting fluorescence imagelight being at least partially outside of the visible spectrum. Thefluorescence image light is captured by the light collection means andpasses through a deformable, variable focal length lens and onto theimage sensor. The focal length of the deformable lens is adjusted suchthat an image captured by the image sensor is in-focus, and thatcaptured image data for the fluorescence image is transmitted to theimage processor. The image processor processes the received images, andthe resulting composite image frames may be displayed sequentially on animage display and/or stored in a computer memory.

According to an inventive method, an inventive medical imaging device isemployed to generate hyperspectral image data, such as a “hyperspectralcube,” the hyperspectral image data including a plurality of individualvideo frames, each with image data collected at distinct and differentfinite wavelength bands. Each collected frame of the hyperspectral imagedata is captured by illuminating a subject scene with a finitewavelength band of light, collecting image light from the scene with ameans such as an objective lens, the collected image light passingthrough a deformable, variable-focal length lens and onto an imagesensor, and adjusting the focal length of the deformable lens such thata captured image is in-focus at the image sensor plane for thewavelength band of light captured. This captured frame is transmitted toan image processor where it may be combined with subsequent capturedframes to generate hyperspectral image data. Subsequent frames arecollected, and the collected wavelength band is varied by one or more ofthe methods described below.

In another aspect of this inventive method, the wavelength band of thesubject scene illuminating light may be changed between each frame, anda corresponding focal length change may be made by adjusting thedeformable lens, such that an in-focus image may be captured by theimage sensor for each subsequent frame at the respective wavelengthband.

In another aspect of this inventive method for generating hyperspectralimage data, the wavelength band may be adjusted by providing a variableband-pass spectral filter upstream of the image sensor, the variableband-pass spectral filter passing only a fraction of the spectrum oflight incident upon it to the image sensor. In one embodiment of theinventive method, the angular position of the spectral filter may beadjusted between frames, and a corresponding adjustment of thedeformable lens may be made, resulting in an in-focus image at a secondwavelength band being able to be captured by the image sensor.

In a preferred embodiment of the inventive method for generatinghyperspectral image data, the collected image light may be split bywavelength range into a first beam with a first wavelength band and asecond beam with a second wavelength band. Subsequent to splitting thebeam, each resultant beam may be passed through a variable band-passspectral filter and corresponding deformable, variable focal length lensto an image sensor. Between each collected frame, the angular positionof the variable band-pass spectral filter for one or more of the opticalchannels may be changed and a corresponding change be made to thedeformable lens in that optical channel, such that an in-focus image atthe passed wavelength band may be captured at the respective imagesensor. In a particularly preferred embodiment, the spectral filters anddeformable lenses may be adjusted at approximately the same time, andthus images may be collected by both image sensors, resulting in two,independent frames being simultaneously captured at two distinctwavelength bands. Collected frame data may be transmitted to an imageprocessing, display and/or storage unit.

In a preferred embodiment, the collected image light may be split bywavelength resulting in one beam of wavelengths less than 1000 nm andanother beam with wavelengths greater than 1000 nm.

Further features of the endoscopic system are disclosed in theco-pending patent application “Medizinische Bildgebungsvorrichtung”(internal file number P18137) filed by the same applicant on the sameday as the present application, which is hereby incorporated byreference into the present application.

The features of the invention as mentioned above and as described belowapply not only in the combinations mentioned but also in othercombinations or alone, without leaving the scope of the presentinvention.

Further aspects of the present invention will be apparent from thefigures and from the description of particular embodiments that follow.These figures show examples of the invention. The figures, thedescription and the requirements contain numerous features incombination. One of skill in the art will recognize the featuresindividually and combine them into meaningful further combinations.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a schematic representation of a medical imaging system in aperspective view;

FIG. 2 shows a cross section diagram of a video endoscope deviceaccording to an example embodiment of the invention;

FIG. 3 illustrates a cross section of an embodiment of the inventionemploying a distal tip imaging system;

FIG. 4 is a schematic representation of another embodiment of theinvention wherein incoming light is split into two beams and at leastone resulting optical channel employs a deformable lens;

FIG. 5 illustrates an embodiment of the invention similar to that shownin FIG. 4 further comprising one or more variable wavelength band filterelements;

FIG. 6 is a representation of a timeline for image acquisition for someembodiments of the invention with a single optical channel;

FIG. 7 is a representation of a timeline for image acquisition for someembodiments of the invention with more than one optical channel andincluding a calibration step for the variable focus optics;

FIG. 8 is a representation of a timeline for image acquisition for someembodiments of the invention involving multiple optical channels andvariable spectral filtration; and

FIG. 9 is a block diagram representing hardware of a system according tosome implementations of the invention.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

FIG. 1 shows a schematic representation of a state-of the art medicalimaging system in a perspective view. The medical imaging system in thiscase is an endoscopic imaging system. Alternatively, the medical imagingsystem could comprise an exoscopic, microscopic or macroscopic imagingsystem. The medical imaging system is intended to examine a cavity. Themedical imaging system comprises at least one medical imaging device 70.The medical imaging device 70 in this case is an endoscope 72.Alternatively, the medical imaging device 70 could be an exoscope, amicroscope and/or a macroscope. The medical imaging device 70 has adistal section 84 designed to be inserted into a cavity during anoperation procedure. The distal section 84 faces the patient during theoperation procedure and faces away from the operator. In addition, themedical imaging device 70 has a proximal section 86 located outside ofthe cavity of the operation procedure. The proximal section 86 facesaway from a patient during the operation and faces towards the operator.The medical imaging device 70 has an intermediate section 88 locatedbetween distal section 84 and proximal section 86. The medical imagingdevice comprises a connection unit 90 located adjacent to intermediatesection 88. The connector unit 90 is formed to connect the proximalportion 86 and the distal portion 84 to each other. In some embodimentsthe connection unit 90 is formed to provide a separable connectionbetween the proximal portion 86 and the distal portion 84, such as abayonet mount.

The medical imaging device 70 has at least one shaft 92. The shaft 92forms at least partially the distal section 84 of the medical imagingdevice 70. The medical imaging device further comprises at least onehandle 94. The handle 94 forms at least partially the proximal section86 of the medical imaging device 70.

The medical imaging device 70 comprises at least one imaging unit 10.The imaging unit 10 may be located in the area of the proximal section86 and be integrated into the handle 94. Alternatively, or additionally,the imaging unit 10 may be located at least partially in the distalsection 84 area, in particular integrated into the shaft 92.

FIG. 2 shows cross section diagram of a video endoscope device 70according to an example embodiment of the invention. This device 70includes a first optical device 208 (in this version “camera head” 208(which may correspond to imaging unit 10 of FIG. 1)) connected in adetachable manner as known in the art, such as with a bayonet mount, toendoscope 72 (“second optical device”). Endoscope 72 includes a shaft92. Other versions may, of course, include a scope integrated with thecamera head or in a single first optical device. Various structuralcomponents supporting the depicted elements are omitted in the figure,as well as other components such as illumination lights sources,fluorescent excitation light sources, optional relay optical elements,and controls, which are known in the art and are not shown in order toavoid obscuring the relevant details of the example embodiments of theinvention.

Toward the left side of the drawing, at the distal tip of the endoscopeshaft 92 is cover glass 211, which in this version faces directly alongthe longitudinal axis of the shaft 92, but may also be positioned at anangle relative to the longitudinal axis as is known in the art. Behindthe cover glass 211 is shown a preferred position for the lens 204, setagainst or very near cover glass 211 and preferably assembled togetherwith the cover glass in construction. While a wide angle lens is shown,this is not limiting and any suitable lens may be used in variousembodiments. Further, the particular number and arrangement of lenses inthe endoscope shaft 92 will vary widely depending on the application.Optically arranged or attached at the proximal side of lens 204 is asecond lens or lens group 209 to focus the incoming light to anappropriate size for the imaging process. The directed light then passesalong endoscope shaft 92, and may be guided by other optical elementssuch as rod lenses. The directed light is received at a doublet lens210, and directed toward first optical device 208, where it passesthrough the proximal window 212 of the endoscope 72 and the distalwindow 213 of first optical device 208. One or more additional lensgroups or rod lenses may be included optically positioned betweendoublet lens 210 and focusing lens group 214. In versions with a unifieddevice, such windows 212, 213 may not be present. Next in the opticalpath is a deformable lens 202, which is deformable to adjust the opticalchannel focal length. The deformable lens 202 can be made of, forexample, birefringent liquid crystal, a transparent elastic membranefilled with fluid, or a two fluid interface. Preferably such aliquid-based deformable lens is employed, but other suitable deformablelens technologies, as are known in the art, may be used. Depicted aretwo different deformable surface configurations for the deformable lens.In the first white light imaging mode, deformable lens 202 is configuredwith the deformable surface at position 215, which generally has morenegative lens power than the second fluoresced light configurationposition 216 of the deformable lens surface. The deformable lenses canbe tuned at least in part by a suitable adjustment mechanism (not shown)such as an electrostatic actuator, an electromagnetic actuator, apiezo-motor, a magneto-strictive actuator, a stepper motor, or anelectroactive polymer actuator for a high focus tuning range, or byother methods known in the art. As depicted, the white lightconfiguration position 215 is a negative power lens and the fluorescedlight confirmation position 216 is a positive power lens, however thisis not limiting, and they may both be negative or both may be positivedepending on the optical channel design. Typically because the chromaticfocal difference between the white light image and the fluoresced lightimage causes a shorter focal distance for the white light, the whitelight mode will have a comparatively negative lens power than thefluoresced light mode. The deformable lens 202 with its adjustmentmechanism is arranged in the first optical device 208 to automaticallyadjust the focus of the first optical device where the automatic focusadjustment compensates for a chromatic focal difference between thewhite light image and the fluoresced light image caused by thedispersive or diffractive properties of the optical materials or opticaldesign employed in the assembly construction of the first or secondoptical devices, or both. The changing of modes, generally automaticallyperformed by the system, in order to provide a real-time overlay,triggers an autofocus algorithm that controls the adjustment of thedeformable lens 202. A priori knowledge of the wavelength ranges to becollected in a single frame, for example by identifying characteristicsof the light source employed by the medical imaging device, can alertthe endoscopic system to the necessary adjustment to the deformable lens202 for each wavelength range to be detected. Preset positions may beemployed for a plurality of modes to achieve the desired configuration,avoiding the need for autofocus processing to determine the desiredfocal length for each mode. It should also be noted that the inventivesystem presented herein is not limited to operating between only twomodes, but may extend to many wavelength ranges, including small, finitewavelength bands, generally of approximately 10 nm bandwidth, oftenemployed in hyperspectral imaging techniques. As is known in the art,hyperspectral imaging divides the spectrum into many individual bands(as opposed to the three general bands perceived by the human eye), andcan extend beyond the visible range into the infrared and ultravioletranges, with the goal of attaining a spectrum distribution of lightreceived by each pixel in a sensor array, and combining theseintensity/spectra maps into a “hyperspectral cube,” generally with thepurpose of identifying materials or processes present in the capturedsubject scene. The present invention does not concern itself withapplications or analysis of the collected hyperspectral cube, but ratherenables the collection thereof in a manner not previously possible.

Next in the optical path is a focusing lens group 214 which in thisversion includes a plano-convex lens and a doublet lens including abiconvex lens and a biconcave lens. Many other suitable lenses andcombinations of lenses may be used. Focusing lens group 214 focuses theimage light toward the image sensor 222 which may include a cover glass.

In one example embodiment image sensor 222 is a single sensor capable ofdetecting both visible light images and fluoresced light images, forexample visible light imagery at approximately 450-650 nm wavelength,and the infrared fluorescence imagery at 800-900 nm. Additionally, oralternatively, the sensor may detect other fluorescent wavelengthscommonly used in endoscopic imagery in addition to the visible lightwavelengths. Because the fluorescent imagery is focused in a differentplane than the visible light, the device has the capability of adjustingthe optical path focus when switching between visible light mode and FImode as described above.

It should be noted that while the position of the deformable lens 202 isshown before the focusing lens group 214 in this embodiment, this is notlimiting, and the deformable lens 202 may be placed in any suitablelocation in the optical channel where the channel construction canaccommodate the varied focal lengths resulting from changing thedeformable lens configuration. For example, deformable lens 202 may bepart of the focusing lens group 214.

While the embodiment described above with relation to FIG. 2, offersmany advantages over the current state-of-the-art endoscopic systemsalready discussed, for example, offering the ability to correctchromatic longitudinal aberration across various spectrum ranges forpre-existing endoscopes to be attached thereto, it should also be notedthat the present invention is not limited to embodiments wherein thedeformable lens is located in a camera head removably attached to anendoscope. FIG. 3, for example, shows an embodiment of the inventionwherein the deformable lens is present in in the distal tip of a videoendoscope. Such distal tip video endoscopes (COTT) can be rigid orflexible scopes. FIG. 3, while not limiting, exhibits the optical systemin an exemplar flexible endoscope. Many of the optical elementscorresponding to those in FIG. 2 are not repeated here. A first opticalgroup 301, located at the distal end 302 of the distal tip 303 of theendoscope, comprises a cover glass and objective lens. In manyembodiments the cover glass is an element of the objective lens andfirst optical group 301. Downstream from the first optical group is thedeformable, variable focus lens 202, that, in a similar manner to thatshown in FIG. 2, is employed to adjust the focal length corresponding tothe viewed wavelength range, in order to provide an in-focus imagefalling on the image sensor 222. The distal tip, flexible endoscopicsystem may also employ a working channel 306 through which tools may beintroduced to the surgical site, and through which material may beremoved therefrom. In addition one or more light guides 307 may bepresent in order to provide illumination at the desired wavelengthranges to the subject scene. The light guides may be attached by andcomprise optical fibers connected to a light source (80 of FIG. 1), asknown in the art. Alternately, distally placed LEDs may be present toprovide illumination. Of course, though not explicitly shown in theother figures, including FIG. 2, other embodiments, including rigidsystems, such as those described in the remaining figures, often alsoinclude light guides and working channels.

A schematic depiction of the optical system of another embodiment isshown in FIG. 4, that may be combined with others described in thisdisclosure, enables the selection of particular wavelength ranges,through the inclusion of tunable spectral filters along the optical pathof one or more beams.

The depicted system uses a dichroic beam splitter 401 to direct a firstbeam of light for example, the visible image light, to a first imagesensor 222 and a second beam of light, for example the fluoresced lightto a second image sensor 402. In this example, fluoresced light entersand is reflected upwards by a dichroic prism interface of beam splitter401 and is incident upon the fluorescence image sensor 402, while thevisible light is passed through the interface 401 to image sensor 222.Beam splitter 401 is preferably a dichroic prism or other suitabledichroic optical element, having a low-pass reflective surface at a 45degree angle allowing higher frequencies to pass through to visiblelight image sensor 222 and lower frequencies to reflect to fluorescencesensor 402. The cutoff frequency may be positioned at or near the topwavelength of visible light, that is near 650 nm, or higher as long asit is below the lowest frequency (higher than longest wavelength)required for detection of the fluoresced light. While the beam splitter401 in this version transmits the visible light wavelengths and reflectsthe fluoresced light wavelengths, this is not limiting and otherversions may pass the fluorescence light and reflect the visible light(with a high-pass dichroic prism, for example), or may reflect both indiffering directions.

Image sensor 222 is cable of detecting the visible light wavelengthscommonly used for endoscopic examination, for example visible lightimagery at approximately the 450-650 nm wavelengths. In this version,image sensor 402 is a single sensor capable of detecting fluorescedlight images, infrared fluorescence imagery, for example, at 800-900 nm.Additionally or alternatively the sensor may detect other fluorescentwavelengths commonly used in endoscopic imagery.

As shown, the fluorescence imaging optical path between the beamsplitter 401 and fluorescence image sensor 402 also comprises adeformable, variable-focus lens 202 in order to adjust the focal planefor the fluoresced light based on characteristics of the attachedendoscope, such as the aberration characteristics or the use of adiffering fluoresced wavelengths in differing scopes for use withvarious fluoresced imaging techniques or to correct for longitudinalchromatic aberration in a distal tip endoscope. The adjustment mechanismfor variable lens 202 may any of those known in the art. The requiredfocal length adjustment provided by the variable focus lens 202 in thisembodiment will typically be less than that required to adjust fordiffering focal lengths between visible and fluoresced light, as thisembodiments adjusts only for different fluoresced light characteristicsof the optical channel such as dispersion and chromatic focaldifferences, or for differences in focal length relating to various FIimaging techniques, all taking place within the wavelength region longerthat of the visible spectrum. This embodiment allows, thereby, thededication of one sensor to the collection of fluorescence images,allowing more light to be captured by, for example, the use of amonochromatic sensor for the fluorescence image sensor 402, obviatingthe need for a Bayer filter on this sensor. This two chip designpermits, as do previously explained embodiments, sequential imaging, andthus near real-time video with the FI image overlaid with the visiblelight image, as will be further discussed below. Alternatively, ofcourse, the images can be displayed separately or as a“picture-in-picture” image on a single display (FIG. 1, 76). Thisembodiment uses deformable lens 202 primarily to adapt to optics alreadypresent in the attached endoscope improving compatibility with an FItechnique used, enabling thereby in-focus overlays of both FI andvisible light irrespective of the optical properties of the attachedendoscope (or permanent optics) and the FI technique being used by asimple calibration, autofocus algorithm, and/or a computation based on apriori knowledge of the associated optics and FI spectrum range.Optionally, a second deformable, variable-focus 403 may be placed in theoptical channel between the beam splitter 401 and the first image sensor222.

A further implementation of the present invention is shown in relationto the embodiment of FIG. 5. This embodiment, and variations thereof,utilize variable optical spectrum filters, as discussed in detail inco-pending application, “Medizinische Bildgebungsvorrichtung” (internalfile number P18137). As in the version shown in FIG. 4, the endoscopicsystem comprises image collecting and (optionally) image relaying optics210 such as an objective, an eyepiece, a lens, a prism, an opticalwaveguide or the like, located upstream of at least one beam splitter401. Viewed in the direction of incidence of the input beam path 500,the beam splitter 401 is arranged downstream from the image collectingand relaying optics 510. The beam splitter 401 splits the input beam 500into at least two beams 520, 530. The beam splitter 401 may split theinput beam 500 into essentially equal parts, or non-equal parts,depending on the desired application. The endoscopic system utilizes atleast two image sensors 222, 402. Each image sensor 222, 402 is designedto capture image information at a frame rate of at least 10 frames persecond. Preferably, the image sensors are designed to capture at least30 images per second. The image sensors in the present case may be CMOSsensors, although other sensors known in the art, such as CCD sensorsmay also be employed. In addition, the image sensors 222, 402, shouldnot be limited to consisting of only a single sensor array for eachsensor, but each image sensor may comprise multiple sensor arrays, thedata from which may be combined, as appropriate, into a single image byan image processor 540.

The endoscopic system of this embodiment comprises at least one opticalfilter unit 506 placed within the optical path 520 between the imagebeam splitter 401 and one of the image sensors 402. The optical filterunit 506 comprises at least one spectral filter. While the opticalfilter may comprise more elements than the spectral filter, we will usethe terms interchangeably in order to improve simplicity of disclosure,with the understanding that the optical filter 506 includes at least aspectral filter. In general, the spectral filter is a band pass filter.In the illustrated case, the spectral filter is separate from the beamsplitter 401. However, in some embodiments the beam splitter 401 canform at least an element of the spectral filter, whereby componentsand/or installation space could be reduced advantageously, as anadditional spectral filter could be dispensed with. The spectral filter506 has at least two angular positions along the optical path 520relative to the image sensor 402. The spectral filter 506 can be toggledbetween the two or more angular positions. Further angular positions, inparticular intermediate positions, of the spectral filter 506 relativeto the image sensor 402 are also possible in various embodiments of theinvention. Each differing angular position results in the transmissionof different wavelength ranges of light to the image sensor 402. Inaddition, the spectral filter 506 may be continuously adjustablerelative to the image acquisition sensor 402. In general, the angularposition of the spectral filter 506 is tilted to change from the firstposition to the second position. The spectral filter 506 can be tiltedabout a swivel axis 516, which is generally parallel to the plane inwhich the filter lies (depicted in this example as orthogonal to theplane of the paper in FIG. 5). As shown in FIG. 5, the swivel axis 516is defined from a side edge of the spectral filter 506, however thisshould not be considered limiting as it is possible that the spectralfilter 506 may be rotatable/tiltable, in particular around a yaw,vertical and/or vertical axis of the spectral filter 506.

The first angular position and the second angular positions of thespectral filter 506 differ in the orientation image sensor 402. Thechange in angle of deviation from the first position to the secondposition is generally at least 2°. This angle will also be not more than45°. As illustrated in FIG. 5, the angle is essentially 25° (originalsensor position indicated by dotted lines).

The optical filter 506 may have at least one support mechanism (notshown) such as a swivel bearing, designed to support the spectral filter506 from the first position to the second position and/or vice versa.The bearing could also be a tilting and/or pivot bearing. The opticalfilter unit 506 has at least one actuator 526. The actuator 526 isdesigned to transfer the spectral filter 506 from the first position tothe second position and/or vice versa. The actuator 526 is also designedto transfer the spectral filter 506 from the second position to thefirst position and back to the second position at least 10 times persecond. Accordingly, the spectral filter 506 is capable of changing itsposition at least 20 times per second, however, ideally, changes on theorder of 30 times or more per second are desired in order to producevideo signals standard to the industry. The actuator 526 may be, forexample, a stepper motor. Steps of the actuator 526 correspond withangular segments of the angle between the first position and the secondposition of the spectral filter 506 relative to the optical path 520between the beam splitter 401 and the image sensor 402.

The pivoting spectral filter 506 in the first position has a firsttransmission range of wavelengths (wavelength band) which it passesthere through to the image sensor 402. The second position has a secondtransmission range of wavelengths which it passes there through to theimage sensor 402. Therefore, by altering the angular position of thespectral filter 506, as described above, various spectral bands can bedetected by the image sensor 402, while others are rejected, not beingpassed through the filter. Thus, the second spectral transmission rangeis at least partially different from the first spectral transmissionrange, and in this way the spectral bands observed by image sensor 402can be selected by the adjustment of the angle of the pivotable spectralfilter 506 relative to the optical path 520 between the image sensor 402and the beam splitter 401.

In order to assure that the image collected at this spectrum is properlyin-focus, variable-focus, deformable lens 202 is positioned, generally,between the spectral filter 506 and the image sensor 402. Additionalnon-varying, focusing and/or directing optics may also be present asseen in the figure. The deformable lens 202, is programmed to adjust thefocal plane based on the known value of the band of the spectrum passingthrough the spectral filter 506 at a given instant. Therefore, bycoordinating the angular position of the filter 506 and the focal lengthof the deformable lens 202, it is possible to ensure an in-focus imageis received for each frame captured by the image sensor 402. For anensuing frame, the position of the sensor 506 may be changed to thesecond position, and the focal length of the deformable lens 202 iscorrespondingly changed in order that the image sensor 402 captures asecond in-focus image at the second transmitted spectral band. Ofcourse, the present invention is not limited to two modes of operation,but enables the collection of as many, distinct, in-focus spectral bandsas may be adequately filtered with a variable position filter asdescribed above. In this matter the invention enables hyper-spectralimaging by collecting images produced in narrow wavelength bands of asingle scene, and offers the added benefit of the ability to ensure thateach wavelength band is collected in the proper focus due to thesynchronized deformable lens. Further discussion regardingsynchronization will be presented below. In one embodiment of theillustration also shown in FIG. 5, the second optical channel 530,between the beam splitter 401 and a second image sensor 222 may or maynot contain a spectral filter unit 507 and/or a deformable lens 40.Embodiments not containing these elements may contain correspondingnon-variable elements, or, may contain one, but not the other element.For example, in a system where the beam splitter 401 passes wavelengthsin the visible spectrum, from about 380 nm to 740 nm, to the secondimage sensor 222, and reflects other wavelengths in the direction of thefirst image sensor 402, it may still be advantageous to employ anadditional fixed wavelength filter in order to filter out undesiredwavelengths, or it may be advantageous to include a deformable lens 403in this optical channel 530 in order to correct any focal plane issuescaused by the optics of a particular attached endoscope. Alternatively.a fixed focal length lens may adequate along this channel, and reducethe complexity and cost of the overall system over one employing onewith a deformable lens along this channel. Such an embodiment could beideal for applications such as FI/visible light video overlays.

While the embodiments represented by FIG. 5 discussed above require onlyone variable spectral filter and one deformable lens, furtherversatility and other advantages are enabled by employing a secondvariable spectral filter 507 (and associated actuator 527) and seconddeformable lens 403 in the second optical channel 530. In particular,this variation offers advantages with regard to hyperspectral imaging.In this case, one sensor can be dedicated to a first range ofwavelengths, while the other sensor is dedicated to a separate, distinctrange, and each optical channel can employ elements, such as specificdichroic filters, particularly chosen to optimize their use in aparticular application. For example, the first spectral filter 506 mighthave an operable variability range between 1000-2000 nm and the secondspectral filter 507 might have an operable variability range between400-1000 nm, and the beamsplitter 401 may be chosen such thatwavelengths over 1000 nm are directed toward the first image sensor 402and wavelengths under 1000 nm are directed toward the second imagesensor 222. Images from each detector may be captured rapidly andsequentially, changing the selected wavelength range passed by each ofthe two filters 506, 507, and adjusting each of the deformable lenses202, 403 between each collected frame, and the resulting image data canbe transmitted to an image processor 540 to generate hyperspectral data,such as a hyperspectral cube, containing the data for each wavelengthband. It should further be noted that in addition to the adjustments inthe optical system between each collected frame described above, theintegration time of the image sensor, or the illumination intensity mayalso be varied, as necessary, to obtain an image of desired exposure,with the understanding that some wavelength bands may, for the sameintegration time, yield images of lesser intensity, and therefore longerintegration times are desired.

Another variation of the invention employs one or more image sensors402, 222 wherein a portion of the individual pixels are filtered bymeans of filter elements associated with individual or groups of pixels.By way of example, the Bayer filter, well known in the art and used inmany color image sensors, filters pixels in sets of four: two greenpixels, one red pixel and one blue pixel, and the resultant data is usedto generate a color image. In contrast, the present invention could use,for example, an image sensor where every other pixel is filtered between1000-2000 nm or between 400-1000 nm, allowing, thereby, the ability todouble the frame rate achievable, by collecting data at these two rangessimultaneously, and utilizing an image processor to generate a hyperspectral image; in this way a single acquisition cycle can collect animage in two spectral ranges. Of course, such a filtered image willtrade off captured image resolution for the multiple filtered images,just as a Bayer filter sacrifices resolution for the ability to generatecolor images. These specialized filters can be used in any of theembodiments presented herein when corresponding image processing isapplied.

Synchronization and timing are of the various components of thedisclosed embodiments are an important part of the present invention.For fluorescent imaging, a light source 80, such as that shown in FIG.1, must provide illumination to the subject scene at a wavelength orrange of wavelengths that result in targeted tissue fluorescing. In thecase of the commonly used ICG FI techniques, excitation radiation in thenear infrared (NIR) spectrum, generally between 600-900 nm, is required,and results in emission radiation with maximum intensity in the range ofabout 830 nm. By contrast, illumination over a wide range from 380-740nm is desirable for visible light imaging. Accordingly, the presentexample implementation synchronizes the light source with the otherelements disclosed above. FIG. 6 illustrates a timeline for acquisitionand presentation of video data for an embodiment of the presentinvention such as those illustrated in FIG. 2 or FIG. 3, wherein asingle optical channel is generally present. At an initial time 601 thesubject scene is illuminated with visible light, simultaneously, beforeor after the start of this illumination period, the visible lightchannel deformable lens is adjusted 602 such that an in-focus image iscaptured 603 by the image sensor. It should be noted, that while theadjustment to the deformable lens may take place before or after theinitial illumination of the images scene with visible light, it must bemaintained in this configuration for the duration of the capture of thevisible light image. The captured image is subsequently transmitted 607to an image processor. Subsequent to the capture 603 of the visiblelight image, the subject scene is illuminated with excitation radiation604 in order to generate fluorescence of the ICG present in the scene.Before, after or during the change in radiation to excitationillumination 604, but after the capture of the visible light image 603,the deformable lens is adjusted 605 such that an in-focus FI image iscaptured 606. Again, it should be noted, that while the adjustment tothe deformable lens may take place before, after or during the initialillumination of the subject scene with excitation light, it must bemaintained in this configuration for the duration of the capture of theFI image 606. Subsequently the captured in-focus FI image is transmitted607 to the image processor. The image processor may then combine orotherwise process the images 608 by means known in the art, and store oroutput the images to a video display 609. Alternatively the images maybe presented on video display as a picture-in picture or side-by side,or other presentation known in the art. Subsequent to the acquisition ofthe FI image, the cycle, may return to step 601 to capture anotherframe. The processed images may be presented on a video display at arate limited only by the time required to perform these steps,preferably at a rate of at least 10 fps, though, more preferably at arate of 30 fps.

In a process similar to that shown in FIG. 6, a system employing asingle optical channel can also be used to perform hyperspectralimaging. In this case the scene could be illuminated by a narrowwavelength band, such as illumination produced by a light emittingdiode, with a bandwidth of, for example, 20 nm. The deformable lens 202,is then adjusted such that an in-focus image is detected and captured bythe image sensor 222. The light source 80 is then adjusted such that thescene is illuminated in an adjacent wavelength band, the deformable lens202 is adjusted such that a second in-focus image is detected andcaptured by the image sensor 222. This process continues over thedesired wavelength ranges, to produce, in the end, a hyperspectral cube,as known in the art, capable of later analysis and/or display.

FIG. 7 shows an inventive synchronization timeline method according tothe dual optical channel medical imaging device embodiment shown in FIG.4. In this process, an endoscope, or other image collecting optics (forexample, in the case of a combined endoscope/image acquisition system,this step might be performed as a final calibration step in theinstrument manufacture procedure), is connected 701 to an imageacquisition system such as a camera head. In order to calibrate thesystem the scene is illuminated with excitation light and deformablelens 202 is adjusted 702 such that an in-focus image is captured at theFI image sensor 402. Optionally, when a deformable lens 403 is alsoincluded in the second optical channel, a similar calibration procedure701,702 can be performed, if necessary, to ensure an in-focus visiblelight image at the associated image sensor 222. Subsequent to thiscalibration step, at a first acquisition time, the scene is illuminatedwith white light 703 and a visible spectrum image is captured 704 at thefirst image sensor 222. The captured visible spectrum image istransmitted 707 to an image processor 540. At a second acquisition time,the scene is illuminated with excitation light 705 and an FI image iscaptured 706 at the second image sensor 402. The captured FI image istransmitted 707 to the image processor 540. It is important to note thatthe first acquisition time may occur subsequent to the secondacquisition time, with the limitation being that the two acquisitiontimes cannot overlap in time, and that when the visible spectrum imageis captured, the scene must be illuminated with white light and when theFI image is captured, the scene must be illuminated with fluorescenceinducing excitation light, and, optimally, the scene should not beilluminated by excitation light and visible light simultaneously, as theintensity of the image produced by the visible illumination generallydrowns out any FI image received due to the overlap of the illuminationspectra. The FI image is captured 706 and transmitted 707 to an imageprocessor 540. The image processor may then combine or otherwise process708 the images, by means known in the art, and store, or output theimages to a video display 709. Alternatively the images may be presentedon video display 76 as a picture-in picture or side-by side, or otherpresentation scheme known in the art. Subsequent to the acquisition ofthe FI image 706, the cycle, may return to step 703 to capture anotherframe. The processed images may be presented on a video display 76 at arate limited only by the time required to perform these steps,preferably at a rate of at least 10 fps, though, more preferably at arate of 30 fps.

With respect to FIGS. 6 and 7, it should be noted that the terms visiblelight and excitation light are used as examples only, and should not beconsidered limiting. Of course, other wavelength ranges could besubstituted in either case as appropriate, for example FI imaging couldbe replaced with a UV analysis, and visible light imaging could bereplaced by a narrow bandwidth color illumination, rather than whitelight.

FIG. 8 represents a timeline for an example hyperspectral imagingprocess with reference to the embodiment shown in FIG. 5, in whichnarrow bands of illumination are imaged on sensors 402 and 222 and acomposite hyperspectral cube is formed from the collected data. In thisexample, the sample scene is illuminated 800 with a broad spectrum ofradiation, for example light with a spectrum from 400 nm to 1000 nm. Thelight is collected 801 by appropriate optics, such as the objective onan endoscope, and transferred to a beam splitter 401 that, in thisexample, splits 802 the beam by wavelength bands, with light of awavelength greater than 1000 nm proceeding along a first optical path520 and light with a wavelength less than 1000 nm proceeding along asecond optical path 530. Along the first optical path, an appropriateangle of the first variable spectral filter 506 is selected andpositioned 813 to permit the passage of only a specific spectral band tothe first image sensor 402. The first variable focus lens 202 isadjusted 814 such that an in-focus image is received at the first imagesensor 222. It should be noted that steps 813 and 814 can happensequentially, simultaneously, or in reverse order. An image of the sceneat a first wavelength range is captured 815 at the first image sensor222 and transmitted 816 to an image processor 540. It should be notedthat step 815 may also include varying the integration time of the imagesensor 222, as appropriate, to acquire an image of sufficient exposureto be useful in as an element of the hyperspectral cube. Simultaneouslyto steps 813, 814 and 815, or previously, or subsequently, along thesecond optical path 530, an appropriate angle of the second variablespectral filter 507 is selected and positioned 823 to permit the passageof only a narrow spectral band to the second image sensor 222. Thesecond variable focus lens 403 is adjusted 824 such that an in-focusimage is received at the second image sensor 222. It should be notedthat steps 823 and 824 can happen sequentially, simultaneously, or inreverse order. An image of the scene at a second wavelength range iscaptured 825 at the second image sensor 222 and transmitted 826 to animage processor 540. It should be noted that step 825 may also includevarying the integration time of the second image sensor, as appropriate,to acquire an image of sufficient exposure to be useful in as an elementof the hyperspectral image data, including the generation of ahyperspectral cube. The image processor 540 may then process the images807 or store, display, and/or transmit them 808. Subsequent totransmission steps 816, 826, the process may return to step 801 for thecollection of subsequent frames of the subject scene, to, generally, becollected at other narrow wavelength bands by subsequent adjustment ofthe filter 506, 507 positions and corresponding adjustment of thevariable focus lenses 202, 403.

It should also be noted that the process shown in FIG. 8 may alsocomprise the step, as an element of Step 800, of illuminating thesubject scene with a narrow band of illumination, rather than a broadspectrum. In this case, techniques known in the art, such as narrow bandLED illumination, may be used to illuminate the scene. Further, aftertransmittal of signals 816, 826 to the image processor, the cycle mayreturn to step 800, rather than 801, wherein the illuminating wavelengthbands are changed by, for example illumination with a new narrowwavelength source, such as a narrow band LED source.

Referring to FIG. 9, a block diagram of system including an imagecapture device according to an example embodiment of the invention isshown. The invention is applicable to more than one type of deviceenabled for image capture, such as endoscopes incorporating solid stateimagers, digital microscopes, digital cameras, mobile phones equippedwith imaging sub-systems. The preferred version is an imaging scopesystem, such as an endoscope.

A light source 80 illuminates subject scene 909 with visible lightand/or fluorescent excitation light, which may be outside the visiblespectrum in the ultra-violet range or the infrared/near infrared range,or both. Light source 80 may include a single light emitting elementconfigured to provide light throughout the desired spectrum, or avisible light emitting element and a one or more fluorescent excitationlight emitting elements. Further, light source 80 may include fiberoptics passing through the body of the scope, or other light emittingarrangements such as LEDs or laser diodes positioned at or near thefront of the scope.

As shown in the drawing, light 910 scattered or reflected from (or,alternatively, as in the case of fluorescence, emitted by) the subjectscene 909 is gathered by an optical assembly 911, where the light isfocused to form an image at a solid-state image sensor(s) and/orfluoresced light sensor(s) 222, 402.

Optical assembly 911 includes at least one lens 204, which may be awide-angle lens element such that optical assembly 911 directs andfocuses light which generally represents a wide field of view. Thedeformable lens 202 (or lenses 202, 403) is (are) part of the opticalassembly. As discussed above, portions of the optical assembly may beembodied in a camera head or other first optical device 208, while otherportions are in an endoscope 72 or other scope device, or the opticalassembly 911 may contained in a single imaging device. Image sensor 222,402 (which may include separate R, G, and B sensor arrays) andfluoresced light sensor 222, 402 convert the incident visible andinvisible light to an electrical signal by integrating charge for eachpicture element (pixel). It is noted that fluoresced light sensor 222,402 is shown as an optional dotted box because embodiments may use asingle sensor 222 to detect both visible light and fluoresced light. Thelatter scheme may be used when the fluoresced light is in a spectrumdetectable by image sensor 222 that is in or near the visible lightspectrum typically detected by a RGB sensor arrays.

The image sensors 222, 402 may be active pixel complementary metal oxidesemiconductor sensor (CMOS APS) or a charge-coupled device (CCD).

The total amount of light 910 reaching the image sensor(s) 222, 402 isregulated by the light source 80 intensity, the optical assembly 911aperture, and the time for which the image sensors 222, 402integratecharge. An exposure controller 940 responds to the amount of lightavailable in the scene given the intensity and spatial distribution ofdigitized signals corresponding to the intensity and spatialdistribution of the light focused on image sensor(s) 222, 402.

Exposure controller 940 also controls the emission of fluorescentexcitation light from light source 80, and may control the visible andfluorescent light emitting elements to be on at the same time, or toalternate to allow fluoresced light frames to be captured in the absenceof visible light if such is required by the fluorescent imaging schemeemployed. Exposure controller 940 may also control the optical assembly911 aperture, and indirectly, the time for which the image sensor(s)222, 402 integrate charge. The control connection from exposurecontroller 940 to timing generator 926 is shown as a dotted line becausethe control is typically indirect.

Typically, exposure controller 940 has a different timing and exposurescheme for each of sensors 222, 402. Due to the different types ofsensed data, the exposure controller 940 may control the integrationtime of the sensors 222, 402 by integrating a visible light sensor 222,402 up to the maximum allowed within a fixed 60 Hz or 50 Hz frame rate(standard frame rates for USA versus European video, respectively),while a fluoresced light sensor 222, 402 may be controlled to vary itsintegration time from a small fraction of visible light sensor frametime to many multiples of visible light sensor 922. The frame rate ofvisible light sensor will typically govern the synchronization processsuch that image frames based on the visible light sensor 923 arerepeated or interpolated to synchronize in time with the 50 or 60 fpsrate of a fluorescence sensor. Alternately, the frame rate of thevisible light sensor may be slowed to match that of a fluorescencesensor.

Analog signals from the image sensor(s) 222, 402 are processed by analogsignal processor 932 and applied to analog-to-digital (A/D) converter924 for digitizing the analog sensor signals. The digitized signals eachrepresenting streams of images or image representations based on thedata, are fed to image processor 540 as image signals 927, 929. Forversions in which a single image sensor 222 also functions to detectmultiple wavelength bands both streams of data are included in the imagesignal 927, typically in one or more of three color channels.

An adjustment control circuit 920 may be provided for supplying thedriving signals to operate the adjustment mechanism for the deformablelenses 202, 403 and variable spectral filters 506, 507 according to thevarious embodiments herein. For versions in which image filter positionsare adjusted, the adjustment control circuit sends appropriate drivingsignals to the mechanical or electrical actuators 526, 527, such as apiezo-electric motor, and may also receive position signals back fromthe actuators. Adjustment control circuitry 920 sends appropriate drivesignals to the deformable lens 202, 403 adjustment mechanism, such as anactuator or piezo-electric motor, and may also receive position signalsfrom the adjustment mechanism. Image processor 540 includes circuitryperforming digital image processing functions to process and filter thereceived images as is known in the art. Image processor may includeseparate, parallel pipelines for processing the visible light image dataand FI image data separately. Such circuitry is known in the art andwill not be further described here. In some embodiments, image processor540 may also perform known autofocus algorithms to allow feedbackcontrol of adjustment control circuitry 920 to compensate for chromaticfocal difference between the white light image and the fluoresced lightimage. However, in preferred embodiments, such adjustments arepredetermined and stored in system memory 956 to allow quick andreliable focus adjustment. In some versions, the predetermined settingsmay be stored in memory in the first optical device 208 itself ratherthan a camera control unit (CCU) 66 or other attached controller.

Image processor 540 may provide algorithms, known in the art, forcombining visible light imagery with FI imagery in a combined imagedisplay, and further highlighting or emphasizing the FI imagery foreasily distinguishing the presence of fluorescing features in the image,or generation of hyperspectral data or other multiple wavelength bandanalysis.

Timing generator 926 produces various clocking signals to select rowsand pixels and synchronizes the operation of image sensors 222, 402,analog signal processor 932, and A/D converter 924. Imaging unit 10includes the image sensors 222, 402, adjustment control 920, the analogsignal processor 932, the A/D converter 924, and the timing generator926. The functional elements of the imaging unit 10 can be fabricated asa single integrated circuit as is commonly done with CMOS image sensorsor they can be separately-fabricated integrated circuits.

The system controller 950 controls the overall operation of the imagecapture device based on a software program stored in program memory 954.This memory can also be used to store user setting selections and otherdata to be preserved when the camera is turned off.

System controller 950 controls the sequence of data capture by directingexposure controller 940 to set the light source 80 intensity, theoptical assembly 911 aperture, and controlling various filters inoptical assembly 911 and timing that may be necessary to obtain imagestreams. In some versions, optical assembly 911 includes an opticalfilter configured to attenuate excitation light and transmit thefluoresced light. A data bus 952 includes a pathway for address, data,and control signals.

Processed image data are continuously sent to video encoder 980 toproduce a video signal. This signal is processed by display controller982 and presented on image display 76. This display is typically aliquid crystal display backlit with light-emitting diodes (LED LCD),although other types of displays are used as well. The processed imagedata can also be stored in system memory 956 or other internal orexternal memory device.

The user interface 960, including all or any combination of imagedisplay 76, user inputs 964, and status display 962, is controlled by acombination of software programs executed on system controller 950. Userinputs typically include some combination of typing keyboards, computerpointing devices, buttons, rocker switches, joysticks, rotary dials, ortouch screens. The system controller 950 manages the graphical userinterface (GUI) presented on one or more of the displays (e.g. on imagedisplay 988). System controller 950 may receive inputs from buttons orother external user interface controls on the scope itself (or softwarecontrols through the GUI) to receive inputs to control the process forautomatically adjusting the focus according to the present invention. Inparticular, the system controller 950 will typically have a mode toggleuser input (typically through a button on the endoscope or camera headitself, but possibly through a GUI interface), and in response transmitcommands to adjust image processing circuitry 930 based on predeterminedsetting stored in system memory. Such settings may include differentsettings for different models of scopes that may be attached to a camerahead or other imaging device containing imaging unit 10.

Image processor 540 is one of three programmable logic devices,processors, or controllers in this embodiment, in addition to a systemcontroller 950 and the exposure controller 940. Image processor 540,system controller 950, exposure controller 940, system and programmemories 956 and 954, video encoder 980 and display controller 982 maybe housed within camera control unit (CCU) 66.

CCU 66 may be responsible for powering and controlling light source 80,imaging unit 928, and/or optical assembly 911. In some versions, aseparate front end camera module may perform some of the imageprocessing functions of the image processor 540.

Although this distribution of imaging device functional control amongmultiple programmable logic devices, processors, and controllers istypical, these programmable logic devices, processors, or controllerscan be combinable in various ways without affecting the functionaloperation of the imaging device and the application of the invention.These programmable logic devices, processors, or controllers cancomprise one or more programmable logic devices, digital signalprocessor devices, microcontrollers, or other digital logic circuits.Although a combination of such programmable logic devices, processors,or controllers has been described, it should be apparent that oneprogrammable logic device, digital signal processor, microcontroller, orother digital logic circuit can be designated to perform all of theneeded functions. All of these variations can perform the same functionand fall within the scope of this invention.

The foregoing has outlined rather broadly the features and technicaladvantages of the invention in order that the detailed description ofthe invention that follows may be better understood. It should beappreciated by those skilled in the art that the conception and specificembodiments disclosed may be readily utilized as a basis for modifyingor designing other structures for carrying out the same purposes of theinvention. It should also be realized by those skilled in the art thatsuch equivalent constructions do not depart from the scope of theinvention as set forth in the appended claims.

Although the invention and its advantages have been described in detail,it should be understood that various changes, substitutions, andalterations can be made herein without departing from the scope of theinvention as defined by the appended claims. The combinations offeatures described herein should not be interpreted to be limiting, andthe features herein may be used in any working combination orsub-combination according to the invention. This description shouldtherefore be interpreted as providing written support for any workingcombination or some sub-combination of the features herein. Moreover,the scope of the present application is not intended to be limited tothe particular embodiments of the process, machine, manufacture,composition of matter, means, methods and steps described in thespecification. As one of ordinary skill in the art will readilyappreciate from the disclosure of the invention, processes, machines,manufacture, compositions of matter, means, methods, or steps, presentlyexisting or later to be developed that perform substantially the samefunction or achieve substantially the same result as the correspondingembodiments described herein may be utilized according to the invention.Accordingly, the appended claims are intended to include within theirscope such processes, machines, manufacture, compositions of matter,means, methods, or steps.

1. A medical imaging system, comprising a medical imaging devicecomprising a shaft and operable to capture a first image with a firstfinite wavelength band of light, and to capture a second image with asecond finite wavelength band of light, wherein the first wavelengthband of light and the second wavelength band of light are different,comprising: a first imaging lens to capture light from an illuminatedsubject scene; an imaging unit, comprising one or more image sensors,operable to capture images in the first wavelength band and the secondwavelength band; a deformable lens located upstream of the one or moreof the image sensors; an adjustment control circuit; and an imageprocessor, wherein that the deformable lens is adjusted by theadjustment control circuit, such that the focus of the medical imagingdevice compensates for chromatic focal difference between the lightreceived by the one or more image sensors at the first wavelength bandand light received by the one or more image sensors at the secondwavelength band, wherein the chromatic focal difference is a result ofthe dispersive or diffractive properties of optical materials or anoptical design employed in an assembly construction of the medicalimaging device, and further wherein the image processor receives a firstimage from the imaging unit and a second image from the imaging unit. 2.The medical imaging system of claim 1 wherein the image processor isoperative to form a first composite image by combining portions of thefirst image and portions of the second image.
 3. The medical imagingsystem of claim 2 wherein the imaging processor is operative to receivea third image from the imaging unit in the first image wavelength band,and to receive a fourth image from the imaging unit in the second imagewavelength band, and to form therefrom a second composite image.
 4. Themedical imaging system of claim 3 further comprising an image displayand/or a system memory and wherein the composite images are displayed onthe image display and/or stored in the system memory.
 5. The medicalimaging system of claim 4 wherein the composite images are displayedsequentially.
 6. The medical imaging system of claims 1 furthercomprising a beam splitter located upstream from the imaging unit, andwherein the imaging unit comprises a second image sensor, and wherein anlight is split by the beam splitter into a first beam and a second beam,and wherein the first beam is incident on the first image sensor and thesecond beam is incident on the second image sensor.
 7. The medicalimaging system of claim 6 wherein the beam splitter splits light basedon wavelength, such that the first beam corresponds to the firstwavelength band and the second beam corresponds to the second wavelengthband.
 8. The medical imaging device of claim 7 further comprising asecond deformable lens located upstream from the second image sensor anddownstream from the beam splitter.
 9. The medical imaging device of anyclaims 8 further comprising a spectral filter located upstream of one ormore of the image sensors.
 10. The medical imaging device of claim 9wherein the image sensor or sensors comprise individual pixels, andwherein the spectral filter comprises filter elements coveringindividual pixels of the image sensor or sensors.
 11. The medicalimaging device of claim 9 wherein the spectral filter is a variable bandspectral filter.
 12. The medical imaging device of claim 11 wherein thewavelength range passed by the spectral filter is determined by theangular position of the spectral filter relative a first optical pathdefined by the deformable lens and the image sensor.
 13. The medicalimaging device of claim 8 further comprising a first variable bandspectral filter located upstream of the first image sensor and a secondvariable band spectral filter located upstream of the second imagesensor, and wherein a first wavelength range passed by the firstspectral filter is determined by an angular position of the firstspectral filter relative to a first optical path defined by the firstdeformable lens and the first image sensor, and whereby the a secondwavelength range passed by the second spectral filter is determined byan angular position of the second spectral filter relative to a secondoptical path defined by the second deformable lens and the second imagesensor.
 14. The medical imaging device of claim 12 further comprising anactuator to vary the angular position of a corresponding variable bandspectral filter.
 15. The medical imaging device of claim 13 furthercomprising one or more actuators to vary the angular position of acorresponding variable band spectral filter.
 16. The medical imagingdevice of claim 5 wherein the images are captured at a frame rate of atleast 10 frames per second.
 17. A method for displaying sequentialcomposite frames of an subject scene generated by a medical imagingdevice, wherein each the sequential frame is a composite of an image ofthe subject scene captured in a first spectrum and an image captured ina second spectrum, wherein at least some portion of both captured imagesis represented in the composite frame; comprising the steps ofsequentially capturing, processing, and displaying a plurality ofprocessed frames, wherein the method of generating each composite framecomprises the steps of; illuminating the subject scene with a firstwavelength band of light in a finite spectrum; collecting image lightfrom the illuminated subject scene with an objective lens; directing thecollected image light to pass through a deformable lens, and to an imagesensor; adjusting the focal length of the deformable lens, such that animage captured by the image sensor is in-focus; capturing in-focus imagedata of the subject scene in the first spectrum; transmitting thecaptured in-focus image data of the subject scene in the first spectrumto an image processor; illuminating the subject scene with a secondwavelength band of light, the second wavelength band of light in asecond finite spectrum; collecting image light from the second spectrumilluminated subject scene with the objective lens; directing thecollected second spectrum image light through the deformable lens and tothe image sensor; adjusting the focal length of the deformable lens,such that an second spectrum image light image captured by the imagesensor is in-focus; capturing an in-focus image data of the subjectscene in a second spectrum at least partially outside of the firstspectrum; transmitting the captured, in-focus image data of the subjectscene in the second spectrum to the image processor; processing thetransmitted images by combining at least portions of the transmittedfirst spectrum captured image data with at least portions of the secondspectrum captured image data to produce a first processed frame; anddisplaying on a display device and/or storing in a computer memory theprocessed frame.
 18. The method of claim 17 wherein the second finitespectrum is chosen to cause at least a portion of the subject scene tofluoresce, and wherein the collected second spectrum image light isfluorescence image light, the fluorescence image light being in awavelength at least partially outside the visible spectrum.
 19. Themethod of claim 18 wherein the portions of the second spectrum capturedimage data are displayed as false colored image data combined with saidfirst spectrum captured image data.
 20. A method for generatinghyperspectral image data by collecting image light with a medicaldevice, the hyperspectral image data comprising a plurality ofindividual video frames, each video frame comprising image datacollected at a distinct and different finite wavelength band of lightwherein the capture of each frame comprises the steps of illuminatingthe subject scene with a finite wavelength band of light; collectingimage light from the illuminated subject scene with an objective lens;directing the collected image light through a deformable lens and to animage sensor; adjusting the focal length of the deformable lens, suchthat an image captured by the image sensor is in-focus; capturingin-focus image data of the subject scene illuminated at the finitewavelength band; transmitting the captured, in-focus image data to theimage processor; and storing the in-focus image data corresponding tothe finite wavelength band.
 21. (canceled)
 22. (canceled)
 23. (canceled)24. (canceled)
 25. (canceled)
 26. (canceled)
 27. (canceled)